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Collaboration  Technologies  and 
Command  and  Control  (C2) 


*  Recent  military  acquisitions  emphasize  introducing 
collaboration  technologies  into  C2  environments 
(Kaufman,  2005) 

*  Personnel  are  expected  to  rapidly  coalesce  into 
functioning  teams  (Boiney,  2005) 

*  Performance  may  be  facilitated  through  emerging 
collaborative  technologies  (i.e.,  email,  IM,  virtual 
whiteboards,  videoconferencing,  etc.)  (e.g.,  Alberts  & 
Hayes,  2003) 


Potential  Problems  with  Collaboration 

Technologies 


*  Bordia  (1997):  Literature  synthesis 

*  Baltes  et  al.  (2002):  Meta-analysis 

*  Concluded  that  teams  restricted  to  text-based 
collaboration  technologies: 

—  Made  poorer  decisions 

—  Took  more  time  to  reach  a  decision 

—  Experienced  less  satisfaction  with  team  processes 

—  Pattern  of  results  was  observed  across  different 
experimental  tasks 

—  Bordia  (1997):  Restricted  communication  impairs 
team  comprehension 


Task  Type  and  Collaboration 
Technologies 


*  Using  McGrath’s  (1984)  circumplex  model,  team 
experiments  can  be  categorized  by  task  type 

•  Studies  reviewed  by  Bordia  (1997)  &  Baltes  et  al. 
(2002)  are  primarily  choosing  tasks 

—  Require  problem  solving  in  situations  with  and 
without  correct  answers 

—  Generally,  task  is  completed  when  the  team 
achieves  a  consensus 


Task  Type  and  Collaboration 
Technologies 


*  C2  tasks  are  better  described  as  execution  tasks 

(McGrath,  1984) 

—  Involve  competition  (both  inter-  and  intra-team)  or 
performance  measured  against  a  standard  of 
excellence 

—  Team  performance  dependent  upon  in-team 
performance  and  opposing-team  performance 

—  Generally,  task  completion  criterion  are  different 


Goals  and  Hypotheses 


*  Goal  for  the  study  was  to  evaluate  the  potential  utility 
of  instant  messaging  (IM)  and  to  examine  its  effects  on 
team  performance  in  an  execution  task  (RoboFlag) 

•  Hypotheses: 

—  Communication  restricted  to  IM  would  result  in  lower 
mission  success  rates,  longer  mission  completion 
times,  and  less  coordinated  team  strategies 

—  Restricted  communication  would  result  in  higher 
workload  and  lower  situational  awareness 

—  Teams  restricted  to  IM  would  send  more  instant 
messages  than  teams  whose  communication  was 
unrestricted 


Method 


*  Participants 

—  36  paid  participants  (28  men,  8  women) 

—  Participants  completed  experiment  in  groups  of 
four,  yielding  a  total  of  nine  experimental  groups 


*  Experimental  design 

—  2  x  3  within-subjects  design 
—  Control  environment  (remote,  co-located) 

—  Level  of  abstraction  (manual,  automated,  mixed)* 
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Method 


*  Participants  completed  six  mission  trials  in  each 
condition  (36  trials  total). 


*  Control  environment  was  a  block  factor  (12  trials  per 
block),  and  level  of  abstraction  was  randomized  within 
each  block. 


*  Participants  filled  out  the  NASA-TLX  and  one  item 
from  the  3-D  SART  following  each  mission  trial. 
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Control  Environment 


Partition 


Remote 


Co-Located 


Arbiter 


RoboFlag  Simulated  Environment 


PLAY 


FUEL 


Destination 
Destination 
Circle  Offense 
Guard  Position 
Guard  Position 
Guard  Position 


ROBOT  ID 

STATE 

TYPE 

Robot  1 

ACTIVE 

CIRCLE 

Robot  2 

ACTIVE 

CIRCLE 

Robot  3 

FLAGGED 

CIRCLE 

Robot  4 

ACTIVE 

TRIANGLE 

Robot  5 

ACTIVE 

TRIANGLE 

Robot  6 

ACTIVE 

TRIANGLE 
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Method 


*  Participants  given  written  and  verbal  instructions 
on  the  capabilities  of  circles  &  triangles 

*  Participants  told  that  experiment  involved  a  game 
similar  to  ‘capture  the  flag’ 

—  Each  team  was  in  direct  competition  with  the 
other 

*  Any  single  trial  continued  until  one  team 
successfully  captured  the  other  team’s  flag 
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Method 


*  Participants  allowed  to  practice  for  five  minutes 
—  Option  of  additional  practice  time  if  needed 

*  Prior  to  the  start  of  each  mission  trial,  participants 
were  allotted  30  seconds  for  communication 

(30  second  “huddle”) 
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Results 


*  RoboFlag  software  recorded  which  team  successfully 
captured  the  flag  (the  winner)  and  the  time  elapsed 
during  each  mission  trial. 

—  Also  recorded  the  number  of  vehicle  position 
changes  initiated  by  each  participant 

*  Analysis  strategy:  Follow  the  Winner 
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Results 


*  Data  for  each  factor  tested  for  statistical  significance 
by  means  of  a  2  (control  environment)  *  3  (level  of 
abstraction)  repeated  measures  analysis  of  variance 
(ANOVA) 

*  For  the  frequency  of  wins,  mission  length,  and  number 
of  vehicle  position  changes  no  statistically  significant 
differences  were  detected  between  the  two  conditions 
(F[1,  8]  =  0.22,  0.49,  0.45  respectively,  p  >  .05). 
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Results 


*  One  possible  explanation  for  the  results  was  that  one 
team  consistently  won  all  mission  trials  (i.e.,  teams 
were  unevenly  matched  -  team  1  vs.  team  2 
distinction). 

*  The  number  of  mission  trials  each  team  won  was 
counted  and  compared  by  means  of  a  two-sample  t- 
test. 

*  Result  indicated  that  there  was  not  a  significant 
difference  for  number  of  wins,  t  (1 6)  =  0.73,  p  >  .05. 
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Results 


*  Data  were  also  examined  to  identify  patterns  of  wins 
that  were  not  due  to  the  experimentally  manipulated 
factors. 

—  Defined  a  win  ‘streak’  as  three  or  more  serial  wins 
by  the  same  team 

—  A  total  of  38  win  streaks  were  identified  in  the  data 

*  Mean  number  of  win  streaks  per  experimental 
session  per  team  was  2.11  (SE  =  0.32) 

*  Mean  number  of  trials  in  a  streak  was  4.05  (SE  = 
0.45) 

—Neither  was  statistically  significantly  different 
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Results 


*  Tested  the  effects  of  the  experimental  conditions  on 
participants’  workload  and  situational  awareness 
ratings  by  means  of  a  2  (control  environment)  *  3 
(level  of  abstraction)  repeated  measures  ANOVA. 

*  For  workload  and  situational  awareness,  no 
statistically  significant  differences  were  detected 
between  the  remote  and  co-located  conditions  (F[  1, 
35]  =  0.30,  0.00  respectively,  p  >  .05). 
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Results 


*  From  IM  logs,  total  number  of  communications  per 
experimental  session  was  calculated. 

*  Messages  were  divided  into  three  categories, 
depending  on  when  they  were  sent: 

—  Pre-game  messages 

—  In-game  messages 

—  Post-game  messages 
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Results 


*  IM’s  sent  between  teammates  were  analyzed  to 
determine  content. 

*  Messages  were  coded  as  either  ‘irrelevant’  (e.g.,  “I’m 
hungry,”  “I  like  this  game”)  or  ‘strategy-relevant’  (e.g., 
“go  straight  for  their  flag,”  “use  more  robots  next 
time”). 

*  Two  coders  separately  classified  each  instant 
message  into  one  of  the  two  categories. 

—  Inter-coder  reliability  was  good  ( Kappa  =  0.92). 
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Results 


*  Mean  number  of  strategy-relevant  and  irrelevant 
instant  messages  sent  during  each  messaging  period 
for  both  command  environments  were  compared 
using  a  2  (type  of  message)  x  3  (messaging  period)  *  2 
(command  environment)  repeated  measures  ANOVA. 
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Mean  NumberoflnstantMessagesSent 


Results 


Discussion 


Previously  hypothesized  that: 

—  CoTmmmigation  restricted  to  IM  wouia-pesuit  in 
lower  m  issiofTSttecess  raieerltynger  mission 
completionJiinesrSnd  les^'oeordinated  team 
lies 

munication  would 

higher  workloadapiUteY.JCAituational 
a' 


y—  Teams  restricted  to  IM  would  send  more  instant 
messages  than  teams  whose  communication 
was  unrestricted 
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Discussion 


*  Overall,  IM  did  not  affect  team  performance. 

*  Contrasts  the  effects  of  collaboration  technologies 
reported  by  Bordia  (1997)  and  Baltes  et  al.  (2002) 

*  Dynamic,  adversarial  nature  of  execution  tasks  may 
favor: 

—  Succinct  messages  between  teammates 

—  Weak  or  generalized  strategies 

—  Feedback  may  engender  dynamic  strategy 
evolution 

*  Supported  by  infrequent  win  streaks 

*  May  explain  high  ratio  of  irrelevant  to  strategy¬ 
relevant  messages 


Discussion 


*  IM  also  did  not  negatively  impact  workload  and 
situational  awareness 

—  Temporal  demands  favor  short  communications 
and  focused  attention 


—  Also,  competition  and  game-related  nature  of  the 
RoboFlag  environment  may  motivate  participation 
(Matthews  &  Westerman,  1994) 
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Discussion 


*  Participants  were  using  IM  for  collaboration 

*  However,  participants  largely  used  IM  for  socialization 
purposes,  rather  than  using  it  exclusively  for  strategy 
development  and  coordination. 

*  May  be  some  concern  on  longer  tasks,  particularly  if 
they  require  less  active  involvement 

—  Potential  for  personnel  to  engage  in  off-task 
conversations  more  frequently,  resulting  in 
distraction,  decreased  situational  awareness,  and 
ultimately  poor  team  performance. 
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Discussion 


*  Current  experiment  offers  limited  support  for  future 
successful  integration  of  collaboration  technologies 
into  command  and  control  environments 

*  Team  performance  unchanged  under  both  command 
environments,  indicating  that  IM  was  at  least  as 
effective  as  face-to-face  collaboration 

*  Results  underscore  need  for  continued  research  into 
team  performance  and  collaboration  technologies  in 
tasks  from  the  executing  quadrant  of  McGrath’s  (1984) 
circumplex  model 
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Discussion 


*  Potential  foci  for  future  research: 

—  Track  strategy  development,  implementation,  and 
execution 

—  Factors  that  mediate  the  use  and  performance 
consequences  of  collaborative  tools  (i.e.,  task 
workload,  time  on  task,  etc.) 
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Questions? 
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